Skip to content

Skip unchanged setWifiOnly writes and put the cause in E_STORAGE - #69

Open
dmurphy5 wants to merge 5 commits into
masterfrom
dylan/storage-cause
Open

dmurphy5 wants to merge 5 commits into
masterfrom
dylan/storage-cause

Conversation

@dmurphy5

@dmurphy5 dmurphy5 commented Nov 10, 2026 •

Copy link
Copy Markdown

What was wrong

In production, E_STORAGE rejects come from iOS only. In 14 days: setWifiOnly 32 users (Sentry DIANA-3427), updateHeaders 17 users, enqueue "cannot save" 16 users. There are two problems.

  1. setWifiOnly() wrote the settings file on every call, also when the value did not change. This was true on both platforms. Diana calls it at every app start. So on a nearly full disk, the call failed at startup for no reason.
  2. Several rejects did not include the cause. setWifiOnly, updateHeaders, pause, resume and the cancel journal path on iOS logged the error with NSLog only. So we could not tell the Sentry events apart.

What changes

Unchanged setWifiOnly. A call with the current value writes nothing and resolves. A changed value writes as before.

  • iOS: setWifiOnly returns early when settings.wifiOnly already has the value. configure() with the same options also writes nothing now.
  • Android: QueueSettingsStore.update skips the write when the transform changes nothing. So on Android, pause, resume and the configure() retry defaults also skip an unchanged write. iOS pause and resume already did this (changeGate).

No partly written tmp file on iOS. When FileIO.writeAtomically fails after it creates the .tmp file (for example ENOSPC during the write), it now deletes the .tmp file. Before, that file stayed and held space that a retry needs. Android already did this.

The cause in every E_STORAGE message. The code stays E_STORAGE. The message text before stays the same. A suffix (cause: …) is added. Diana reads only the code (refusalCode), so no decision in Diana changes. Only the Sentry text changes.

iOS sites and their new message shape:

Site Message
setWifiOnly setWifiOnly: cannot save the queue settings (cause: <domain> <code>; POSIX <n>; <description>)
updateHeaders updateHeaders: cannot save the queue settings (cause: …same…)
pause, resume <method>: cannot save the queue settings (cause: …same…)
cancel, live entry, journal fails cancel: cannot journal the outcome of '<id>'; nothing changed (cause: …same…)
cancel, settled entry, delete fails cancel: cannot delete '<id>': <description> (cause: <domain> <code>; POSIX <n>)
enqueue, save fails (2 sites) enqueue: cannot save '<id>': <description> (cause: <domain> <code>; POSIX <n>)
enqueue, staging fails (5 sites) enqueue: <step>: <description> (cause: <domain> <code>; POSIX <n>)
enqueue, form body cannot be created enqueue: cannot create the form body (cause: <domain> <code>; POSIX <n>; <description>)
enqueue, catch-all (cannot happen today) <description> (cause: <domain> <code>; POSIX <n>)

POSIX <n> comes from the NSUnderlyingErrorKey chain. It is left out when the error is already in NSPOSIXErrorDomain or when there is no POSIX error. The description is left out when the message already has it.

Android sites:

Site Message
enqueue, staging fails could not stage the body: <message> (cause: <class>; errno <NAME>)
enqueue, save fails (2 sites) could not save the entry: <message> (cause: <class>; errno <NAME>)
cancel, journal fails could not journal the cancel: <message> (cause: <class>; errno <NAME>)
UploaderModule.onQueue catch-all (setWifiOnly, updateHeaders, pause, resume, and others) <message> (cause: <class>; errno <NAME>)

errno <NAME> appears only when the cause chain holds an android.system.ErrnoException. QueueException now has a cause. The catch-all still rejects E_STORAGE.

Android rename and directory create. AtomicFiles used File.renameTo and File.mkdirs, which return only a Boolean. It now uses Files.move(..., ATOMIC_MOVE) and Files.createDirectories. Their exceptions say why, for example Permission denied. I did not use android.system.Os.rename, which would also put an ErrnoException in the chain: Os is a stub in the JVM unit tests ("Method rename in android.system.Os not mocked"), so every test that writes a file would fail.

How iOS gets an errno. FileManager.createFile returns only a Bool. Apple does not document that it sets errno. So FileIO.writeSynced and the multipart writer now create the file with POSIX open(O_WRONLY|O_CREAT|O_TRUNC, 0644). A failure throws an NSPOSIXErrorDomain error. FileIO.rename now throws the same kind of error. Its text still has errno <n>.

I did not add a structured userInfo.cause. A comment in FileIO.cause and StorageCause names it as a later option.

Where to look

  • iOS: FileIO.cause, FileIO.create, FileIO.posixError, FileIO.writeAtomically in ios/FileIO.swift. configure, setWifiOnly, changeGate, updateHeaders, cancel and saveSettings in ios/QueueCoordinator.swift.
  • iOS: settle in ios/QueueCoordinator+Outcomes.swift has two forms now. settle(id, outcome) never throws. settle(id, outcome, requireJournal: true) throws the journal error, so cancel can put it in the message. EventJournal.appendOrThrow is the throwing form of append.
  • Android: StorageCause.kt (new), QueueSettingsStore.update, AtomicFiles.writeAtomically, the QueueException wraps in QueueController.kt, and onQueue in UploaderModule.kt.
  • README: one sentence under setWifiOnly, and one next to the codes: E_STORAGE messages are for logs, so match on code.

Risks

  • Sentry grouping. Where Diana uses default grouping, a new message text can split or merge issues. JS errors usually group by stack, so I expect no change. Not checked.
  • File protection on iOS. A file made with open() gets the app's default data protection class, as createFile with no attributes does.

Test plan

  • cd ios && swift test: 245 tests, 0 failures. Before: 235. New in ios/Tests/StorageCauseTests.swift:
    • The same setWifiOnly value writes once. The test deletes settings.json after the first call and checks that the second call does not create it again.
    • A changed value still writes.
    • The same value resolves when the store directory is read-only.
    • Two configure() calls with the same options write once.
    • A write that fails after the .tmp file exists leaves no .tmp file. The test sets a file size limit (RLIMIT_FSIZE, with SIGXFSZ ignored), so write(2) fails with EFBIG part way, as ENOSPC would.
    • setWifiOnly, updateHeaders, pause and resume on a read-only store reject with … cannot save the queue settings (cause: NSPOSIXErrorDomain 13; cannot create settings.json.tmp….
    • The live cancel with a read-only journal rejects with the cause.
    • An enqueue that cannot make its entry directory ends with (cause: NSCocoaErrorDomain 513; POSIX 13).
    • A form enqueue whose multipart file cannot be created rejects with enqueue: cannot create the form body (cause: NSPOSIXErrorDomain 13; ….
    • The formatter with a nested POSIX error.
    • With the fix removed, each of these fails: the 2 unchanged setWifiOnly tests, the configure test and the .tmp test.
  • Android :react-native-background-upload:testDebugUnitTest: 337 tests, 0 failures. Before: 327.
    • New tests in QueueControllerTest cover the same setWifiOnly cases, the catch-all message of a failed settings write, and the cause on a failed cancel journal write. With the skip in update removed, the 2 unchanged-value tests fail.
    • New StorageCauseTest covers the formatter, including ; errno EACCES for an ErrnoException in the cause chain. On the JVM the android.jar stub leaves errno at 0 and cannot name it, so the test replaces the StorageCause.errnoNameOf seam. The default reader (ErrnoException.errno plus OsConstants.errnoName) runs only on a device, so no unit test covers it. It shows on an emulator with a read-only queue directory.
    • A new QueueStoreTest checks that a failed rename throws a FileSystemException and leaves no .tmp file.
    • UploaderModule has no unit tests. The test checks StorageCause.rejectMessage, which onQueue calls.
  • yarn lint:ci, yarn typecheck, yarn jest (194 pass). This change touches no JS.

🤖 Generated with Claude Code

dmurphy5 and others added 2 commits November 10, 2026 11:48
setWifiOnly() wrote the settings file on every call, also when the value
did not change. The Diana app calls it at every start, so on a nearly
full disk the call rejected E_STORAGE for no reason. Now a call with the
current value writes nothing and resolves, on both platforms. On Android
the skip is in QueueSettingsStore.update, so pause, resume and the retry
defaults also skip a write that changes nothing, as iOS pause and resume
already did.

Several E_STORAGE rejections did not say why the write failed, so the
Sentry events could not be told apart. Each one now keeps its message
text and adds " (cause: ...)": on iOS the NSError domain and code, the
POSIX code when there is one, and the description when the message does
not already hold it; on Android the exception class and the errno name
when the cause chain holds an ErrnoException. QueueException now has a
cause. The code stays E_STORAGE.

To get an errno on iOS, FileIO.writeSynced and the multipart writer now
create the file with POSIX open() instead of FileManager.createFile, and
FileIO.rename throws an NSPOSIXErrorDomain error.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Review follow-ups.

iOS configure() wrote the settings file on every call. Now the same
options write nothing, as on Android.

iOS FileIO.writeAtomically left a partly written .tmp file when the write
or fsync failed, for example on a full disk. That file held space that a
retry needs. Now it is deleted, as Android already did.

Android AtomicFiles used File.mkdirs and File.renameTo, which return only
a Boolean, so a failure did not say why. It now uses java.nio
Files.createDirectories and Files.move with ATOMIC_MOVE, whose exceptions
carry the reason. android.system.Os would also give the errno, but it is a
stub in the JVM unit tests, so every test that writes a file would fail.

StorageCause gets a seam for the errno name, so a JVM test can check the
"; errno EACCES" text. New iOS test for the multipart writer when it cannot
create its file. README notes that the same setWifiOnly value resolves
without a write, and that E_STORAGE messages are for logs only.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@dmurphy5
dmurphy5 added this pull request to stack #73 November 10, 2026 18:33
@dmurphy5
dmurphy5 marked this pull request as ready for review November 10, 2026 20:49
@elliottkember
elliottkember requested a review from a team November 12, 2026 17:13
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
dmurphy5 pushed a commit that referenced this pull request Nov 13, 2026
…#80)

## What could go wrong

On iOS, a retry's backoff exists only as the begin date of a background
task: the library creates the next attempt right away and lets
`nsurlsessiond` hold it until the date. If the daemon refuses the app's
connection, that task fails immediately with `NSURLErrorDomain -1`,
before its begin date and without sending anything. The library counted
that as a transient failure and created the next delayed task at once,
which failed the same way.

That loop ran at 300–600 attempts a second, each writing the entry and
opening a new XPC connection. The app sat at about 200% CPU and grew to
gigabytes of memory. We hit it on two simulators running diana's CI dev
builds, which had no entitlements, so the daemon logged "does not have a
bundle ID, rejecting connection" for every session. That build is fixed
in openspacelabs/diana#9825. The library should still not spin when the
daemon says no.

## What this PR does

- **A refused delayed task isn't an attempt.** A delayed task that ends
before its begin date, with no response and no bytes sent, never reached
the daemon. It emits no event and keeps its ordinal and request id. The
app waits out the rest of the delay itself, then issues the attempt
again (`advanceAttempt: false`). All three conditions are needed, so a
clock change during the wait can't turn a real failure into this. The
error code isn't checked, because `-1` also comes with real failures.
- **Chunked parts get the same treatment.** A refused part task keeps
its begin date and refills once the date passes. Unlike a simple
request, the refilled part still takes a new ordinal, as every part task
does.
- **A floor as a backstop, simple requests only.** A retry never starts
within `baseMs` of the previous attempt's start. One that ended before
its own start gets no task for `baseMs`. The floor lives in memory only.
- **`TaskCompletion` carries `bytesSent`** (from `countOfBytesSent`) so
the completion path can tell.

Android doesn't have this flaw: it waits out the backoff itself before
making the request.

## Review notes

- **System cancels change slightly.** If iOS cancels a delayed task
before its begin date with nothing sent, the library now waits out the
delay under the same ordinal, instead of minting a new attempt with a
fresh backoff. It never ran, so that seems right.
- **Four existing tests now advance the fake clock before completing a
delayed task.** They delivered a server response before the task's begin
date, which the real daemon can't do, and that tripped the floor.
- **Conflicts with #69 and #72 in `QueueCoordinator+Outcomes.swift`.**
It's small: one added line, `attemptStarts[id] = nil`, next to code #69
rewrites. #70 and #65 merge cleanly with this branch and pass the suite
(293 and 272 tests).

## How it was tested

- `swift test` in `ios/`: 263 tests, 0 failures, 5 new. They cover every
task refused for 10 s of fake time (bounded attempts, no tight loop),
one refused task keeping its ordinal and X-Request-Id, a task failing
after its begin date still counting, a task that sent bytes or got a 503
early still counting, and a refused chunked part. With the fix disabled
the new tests fail.
- **In diana on an iPhone 15 simulator**, with this branch in
`node_modules` and the app built with `CODE_SIGNING_ALLOWED=NO`, so the
daemon refuses it:
- A stuck upload went from about 5,500 attempts per 30 s to 1 attempt
per backoff.
- Two new uploads backed off with growing gaps (about 14 s, 32 s, 74 s).
  - CPU stayed under 2%.
  - With v11.1.0 the same build spun at 199% CPU.
- No JS changed, so `yarn lint:ci`, `typecheck` and `test` weren't run.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Brings in #65, #74, #78 and #80.

CHANGELOG.md: kept every entry; master's entries first, then this branch's
two (setWifiOnly() no-op write, E_STORAGE cause).
ios/QueueCoordinator+Outcomes.swift: kept this branch's throwing settle()
and added #80's `attemptStarts[id] = nil` after `throttle.reset(id)`, where
master has it. Dropped master's `return true`, since settle() here returns
nothing. A user cancel whose journal write fails still throws before any
change, as master returns false before it.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
dmurphy5 added a commit that referenced this pull request Nov 13, 2026
Brings in the updated base, which now has #65, #74, #78 and #80 from
master.

CHANGELOG.md: kept every entry; master's entries first, then #69's, then
this branch's four.
ios/QueueCoordinator.swift: kept both in init. This branch's header
generation raise runs first, then master's recordBootSnapshot() call; the
recordBootSnapshot() function from master is kept as is.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Brings in #71 (mutate() waits, onWarning).

CHANGELOG.md: kept every entry; master's entries first, then this branch's
two.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
dmurphy5 added a commit that referenced this pull request Nov 13, 2026
Brings in the updated base, which now has #71 from master. No
conflicts. CHANGELOG.md auto-merged with master's entries first, then
#69's, then this branch's.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant